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I. INTRODUCTION 


A. OBJECTIVES 

The United States Navy has conducted a one-year test 
period of the Red/Yellow/Green (RYG) Program which concluded 
on 1 July 1990. The RYG source selection improvement program 
uses contractor past quality performance in the source 
selection process. The program was developed in an attempt to 
reduce the problem associated with poor contractor product 
quality. These poor contractor product quality problems not 
only impair fleet readiness, increase costs, and compromise 
safety but also inhibit the Government from obtaining the best 
purchase value from the contractor. [Ref. 1] 

The RYG Program is designed to use information contained 
in the Navy’s Contractor Evaluation System (CES) and the 
Product Deficiency Reporting and Evaluation Program (PDREP) 
data base. This centralized data base was established at the 
direction of the Secretary of Navy to provide: 

A product deficiency reporting and data feedback system, 
maintenance of contractor/supplier quality history and 
effective use of these data to influence the pre-contract 


award process and formulate the basis for necessary post- 
award quality assurance action. [Ref. 2] 


The CES/PDREP data base is composed of contractor quality 
information gathered from the following sources: 
1. Quality Deficiency Reports (QDRs) 
2. Material Inspection Record (MIR) 
3. Reports of Discrepancy (RODs) 
4. Navy Vendor Data Analysis Report (VDAR) 
5. Pre-award Surveys 
6. Defense Logistics Agency (DLA) Contractor Improvement 
Program (CIP) Alert List 

7. Method C, D, and E Corrective Action Listing 

8. Defense Contractor Management Command (DCMC) Quality 
Systems Reviews 

9. Small Business Administration Certificates of 
Competency. 

These reports are forwarded to NMQAO via their respective 
chain of command. NMQAO then evaluates the reports to 
determine contractor liability and adds the appropriate 
entries to the CES/PDREP data base. Based on the information 
contained in the CES/PDREP data base, RYG classifies each 
contractor as Red (high risk), Yellow (moderate risk) and 
Green (low risk) performers. Contractors who do not meet 
established criteria for RYG classifications are listed in the 
"Insufficient Data" category. Classification is done by 


Federal Supply Classification (FSC) so a contractor who 


produces material in more than one FSC may have more than one 
RYG classification. [Ref. 2] 

The RYG concept combines CES/PDREP contractor quality 
history with prescribed procedures to find the best value. 
RYG emphasizes contractor quality history by adding the cost 
of receiving poor quality goods or services into the 
procurement source selection process. One method of 
accomplishing this costing procedure is through the use of 
Technical Evaluation Adjustment (TEA) which exemplifies the 
expected cost to the Government to correct or take appropriate 
action due to unsatisfactory contractor performance. 

Another method used with the Fixed Price-Greatest Value 
method of procurement is to rate the offerors in term of 
expected quality of performance. [Ref. 2] All proposals 
including necessary TEAS are evaluated to determine a source 
selection that will result in the best overall contract for 
the Government. 

The focus of this research is to analyze the feasibility 
of implementing the RYG Program throughout the Navy 
procurement system. This will include evaluation of the 
benefits derived from utilizing the RYG Program, impediments 
to implementation, and the ability to integrate the RYG 
Program with current procurement procedures. These data will 


include information obtained during the test of RYG Program 


conducted at five Navy field activities (Naval Air Engineering 
Center Lakehurst, Naval Avionics Center Indianapolis In, Naval 
Ships Parts Control Center Mechanicsburg Pa (Code 021, Level 
Ess) Naval Supply Center Charleston/Naval Shipyard 
Charleston, and Naval Supply Center Pensacola/Naval Aviation 


Depot Pensacola) which was compiled by NMQAO. [Ref. 3] 


B. RESEARCH QUESTIONS 

The primary research question is: Should the Navy 
implement the RYG Program for all acquisition / procurement? 
The research will evolve around the actual test data in 
determining the success of this program. 

The secondary research questions in this area are: 
(1) Is the versatility of the RYG Program sufficient to meet 
the requirements of Navywide Procurement? 
(2) Can the RYG Program be integrated with current procurement 
PEaceiceas 
(3) Can the current RYG Program be expanded to include all the 
contractor quality history data for the entire navy 


procurement system. 


C. RESEARCH METHODOLOGY 
Initial research will be conducted by reviewing and 
analyzing data collected from primary sources, including 


actual raw contract award data from each of the test sites. 


These data will be collected specifically through telephone 
interviews and monthly status reports of the RYG test to 
evaluate the current status of the test and the potential for 
future expansion of the program. The interviews will involve 
personnel from Assistant Secretary of the Navy (Research, 
Development, and Acquisition) Reliability, Maintainability, 
and Quality Assurance (ASN(RDA)RM&QA), Naval Material Quality 
Assessment Office (NMQAO), Naval Supply Systems Command 
(NAVSUP), Navy Ships Parts Control Center (SPCC), the five 
activities involved in the test, as well as hardware systems 
commands, and other procurement activities which may be 


involved in the implementation of RYG. 


D. SCOPE AND LIMITATIONS 

The scope of the thesis will be to evaluate the 
possibility of implementing the RYG Program. The scope will 
be limited to the evaluation of RYG procedures, the ability of 
RYG to be integrated with current procurement procedures, and 
the applicability and benefits of RYG for different types of 


pr OCUreEMENRE . 


E. ORGANIZATION 


The remainder of the thesis is organized into the 


following chapters: 


1. Chapter II "Background" provides an understanding of 
the RYG Program and how it interfaces with CES/PDREP. 

2. Chapter III "Analysis of the Red/Yellow/Green Program" 
provides an analysis of the RYG program computer 
capabilities, the test performance, cost and savings 
analysis, integration analysis, and alternatives analysis. 
3. Chapter IV "Impediments and Benefits of 
Implementation" provides the pros and cons of Navywide 
implementation of the Red/Yellow/Green Program. 

4. Chapter V "Conclusions and Recommendations" discusses 
whether or not to implement the Red/Yellow/Green Program 


Navywide and the basis for the decision. 


F. SUMMARY OF FINDINGS 

The findings include the evaluation of the benefits 
provided by a comprehensive analysis of the feasibility of 
implementing the RYG Program throughout Navy procurement 


activities. 


IZ. BACKGROUND 


A. RED/YELLOW/GREEN PROGRAM 

The Navy, like other Department of Defense components, 
continues to experience problems with contractor product 
quality. These problems impair fleet readiness, increase 
costs, and compromise safety. A key to improving quality is 
to use contractor product quality history in the contract 
award process to ensure the Navy receives the quality it 
requires. [Ref. 4] 

The Navy developed the Red/Yellow/Green (RYG) Program to 
meet the requirements of the Department of Defense (DoD) and 
Secretary of Navy (SECNAV) policies. These policies state 
that contractor quality history will be collected and 
maintained in a centralized data base to assure that contracts 
are not awarded to contractors with a previous history of 
providing unsatisfactory quality products without determining 
required quality assurance action prior to and after contract 
award. | 

Red/Yellow/Green Program is the title given to the 
methodology of evaluating and categorizing contractor quality 
performance data by Federal Supply Classification (FSC) and 


using these data to assist in the source selection process. 


Under the RYG Program, a contractor’s past quality performance 
is evaluated and assigned a color classification based upon 
the degree of risk to the Government of receiving poor quality 
products. The RYG program does not classify contractors, but 
rather it classifies the contractor’s quality performance by 
FSC, so a contractor who provides material in more than one 
FSC may have more than one RYG classification. 

The RYG program color classifications are: RED - High 
quality risk, YELLOW - Moderate risk, and GREEN - Low risk. 
Contractors for which there is insufficient data are assigned 
an "Insufficient Data" status. The general description of 
each color classification as outlined in the program are: 

1. RED: The performance history of the contractor for a 
given commodity indicates that he has supplied goods or 
services of poor enough quality to require the application of 
special quality assurance actions. The seriousness of the 
contractor’s negative quality history is sufficient to require 
review and approval by the head of the contracting office (as 
defined by the Federal Acquisition Regulation (FAR)) prior to 
contract award. The contractor is designated as a high 
quality risk. 

The red classification will not be utilized to bar a 
contractor from competing. The intent is to deter awards from 


continually poor performers and ensure that sufficient 


oversight is in place in the event that a red contractor 
receives an award. [Ref. 5] 

2. YELLOW: The performance history indicates’ the 
contractor has supplied goods or services of a particular 
commodity of poor enough quality to require special quality 
assurance actions in an effort to reduce the risk of delivery 
Sp) peor “Quality products ‘to the Navy. The contractor is 
designated as a moderate quality risk. [Ref. 5] 

3. GREEN: The performance history indicates that the 
contractor has supplied goods or services which meet or exceed 
the quality requirements of the contract. His proposal is to 
be evaluated in accordance with established acquisition 
regulations without anticipating special quality actions. The 
contractor is designated as a low quality risk. [Ref. 5] 

The specific criteria used to classify a contractor as 
Red, Yellow, or Green are listed in APPENDIX A. 

4. INSUFFICIENT DATA: A contractor is identified as 
having "Insufficient Data" to meet the RYG classification on 
a particular commodity if: (a) The contractor is a first time 
offeror for that FSC, (b) no quality history is available on 
the contractor for that FSC, (c) the only available quality 
information data are beyond the evaluation periods set forth 
in APPENDIX A. In the case of a contractor being classified 


as having "Insufficient Data", the Contracting Officer may 


elect to employ additional quality assurance actions. 
Technical Evaluation Adjustments (TEAs) will not be added to 
the contractor’s price during the pre-award evaluation 
process. [{Ref. 5] 

It is important to understand that procedures set for the 
RYG Program are not designed to eliminate the requirement that 
a determination of responsibility be made for every 
prospective contractor prior to award. The colow 
classification of a contractor alone is insufficient to 
determine responsibility of the contractor. Responsibility 
determination must be made in accordance with Federal 
Acquisition Regulation 9.104 without consideration of the 
contractor’ s color classification. 

The solicitation documents and synopsis in the Commerce 
Business Daily for procurement that will be made under the RYG 
Program during the test period are required to advise 
contractors of RYG procedures and will indicate that final 
contract award will be based upon a combination of factors, 


including price and historical quality performance. 


B. CONTRACTOR EVALUATION SYSTEM/PRODUCT DEFICIENCY REPORTING 
AND EVALUATION PROGRAM 
The RYG program uses information contained in the Navy’s 


Contractor Evaluation System (CES) and the Product Deficiency 
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Reporting and Evaluation Program (PDREP) data base. The 
CES/PDREP are managed by the Naval Sea Systems Command 
detachment, Naval Material Quality Assessment Office (NMQAO), 
under the direction of the Office of the Assistant Secretary 
of the Navy (Research, Development, and Acquisition) 
Reliability, Maintainability, and Quality Assurance 
(ASN (RDA) RM&QA) . 

The data base 1S composed of contractor quality 


information gathered from the following sources: 


1. Quality Deficiency Reports (QDRs). QDRs are 
prepared by Navy field activities to document product 
quality deficiencies, design deficiencies, OG 
inadequate procurement documents resulting cig 
defective new and newly reworked material being 
delivered to the Navy. (Ref. 5] All QDRs_ are 


submitted to the Naval Air Systems Command (NAVAIR), 
the Navy focal point for QDRs. Once each week, QDRs 
determined to be contractor liable and with defects 
verified, are transmitted by NAVAIR to NMQAO for 
inclusion in the PDREP data base. [Ref. 6] 


2. Material Inspection Record (MIR). MIRS are 
prepared either by Navy representatives performing 
technical inspections at a contractor’s plant or by 
Navy field activities performing technical inspections 
upon receipt of material. MIRS are submitted to the 
Navy Systems Command having cognizance over the field 
activity. [Ref. 3] The Systems Commands (NAVAIR, 
NAVSUP, Naval Facilities Engineering Command, and 
Space and Naval Warfare Command) then transmit the 
MIRs to NMQAO. The MIRS generated by NAVSEA 
activities are submitted directly to NMQAO. [Ref. 6] 


3. Reports of Discrepancy (RODs). RODS are prepared 
by Navy field activities to document receipt of 
incorrect material, shortages and overages, and 
discrepancies in preservation, packing, and marking. 
RODs are submitted to Naval Supply Systems Command 
(NAVSUP). [Ref. 5] However due to lack of real value 


dig 


of the RODs to the RYG program, RODS are no longer 
being included in the classification process; however, 
they will continue to be collected for the CES/PDREP 
program. [Ref. 7] 


4. Defense Logistics Agency (DLA) Contractor 
Improvement Program (CIP) Alert List. Contractors are 
placed on the DLA alert list if DLA has placed them in 
the CIP, if they have received a negative pre-award 
survey, or if Defense Contract Management Command 
(DCMC) has recommended they be given a pre-award 
survey for a particular reason. (Ref. 8] DLA 
sends a hard copy of the list to ASN(RDA) RM&QA. ASN 
then sends a copy to NMQAO.[Ref. 6] 


5. Navy Vendor Data Analysis Report (VDAR). The VDAR 
identifies contractors who, because of past poor 
performance, should be considered carefully before 
being awarded a contract and should be monitored after 
contract award. Evaluation of performance is based on 
data from pre-award surveys, QDRs, open DLA method C, 
D, or E corrective action, and conviction or an 
investigation for malpractice or fraud. [Ref. 8] The 
VDAR is compiled by NMQAO based on past performance 
and input from Navy Systems Commands and their field 
activities. [Ref. 6] 


6. Pre-award Surveys. Pre-award surveys are conducted 
by contract administration offices when a procuring 
contracting officer needs additional information to 
determine contractor's management, financial 
capability, and technical skill to determine whether 
he/she will be able to perform the proposed contract. [Ref. 9] 
Only those pre-award surveys requested by Navy 
activities are included in PDREP. The Navy activities 
that requested the survey submit a copy of the 
completed pre-award survey to the cognizant Systems 
Command. The Systems Commands then transmit copies to 
NMOQAO. NAVSEA activities submit copies of surveys 
directly to NMQAO. [Ref. 6] 


7. Method C,D, and E Corrective Action Listing. 
Contractors are placed on the corrective action 
listing if DLA has documented deficiencies in their 
quality programs. Specifically, method C indicates 
that the contractor has a serious quality problem or 
has not corrected a deficiency documented using method 
B (a major deficiency). The Government sends a letter 


eZ 


to the firm’s top management requesting corrective 
action. Method D indicates that less severe methods 
of corrective action (i.e., A, B, and C) have failed. 
The acquisition quality assurance program is 
discontinued, and the contractor is advised that the 
Government will not accept his goods or services until 
deficiencies have been corrected. Method E is used to 
advise a prime contractor that a subcontractor has 
quality deficiencies that would justify method C or D 
corrective action in a prime contractor and to request 
that the prime take corrective action with his 
subcontractor. {[Ref. 10] DCMC sends a hard 
copy of the listing to ASN(RDA) RM&QA. ASN then sends 
a copy to NMQAO. [Ref. 6] 


8. Product-Oriented Surveys. Product-Oriented surveys 
are technical product inspections conducted in a 
contractor’s plant when a buying activity desires to 
perform a special test on an item. They are performed 
by DCMC when requested by the buying activity. Alig 2 
DCMC does not have the necessary resources, the buying 
activity may perform the survey. When a Navy activity 
requests a product-oriented survey, it submits a copy 
to the appropriate System Command. The Systems 
Commands then transmit the surveys to NMQAO. NAVSEA 
activities submit copies of surveys directly to NMQAO. 
[Ref. 6] 


9. Defense Contract Management Command (DCMC) Quality 
System Reviews. Quality system reviews are performed 
by DEMG. They involve an evaluation of the 
contractor’s quality procedures and verification that 
the contractor’s quality practices conform to those 
procedures. [Ref. 10] The reviews also evaluate the 
Government’s in-plant quality assurance program. Navy 
activities receive copies of quality system reviews if 
they participate in the review with DCMC or if they 
request a copy. Copies received by Navy activities 
are submitted to the appropriate Systems Command. The 
Systems Commands then transmit the reviews to NMQAO. 
NAVSEA activities submit copies directly to NMQAO. 
[Ref. 6] 


10. Small Business Administration Certificates of 
Competency (COC). If a small business is determined 
to be non-responsible by a Government buying activity, 
the small business can request that the Small Business 
Administration (SBA) determine whether the business is 


r 


responsible. If the SBA concludes that the small 
business is responsible, it will prepare a COC to 
document that determination. The buying activity must 
then treat the small business as a responsible 
offeror. [Ref. 3] The SBA sends hard copies of COCs 
to NMQAO for inclusion in PDREP. COCs are collected 
mainly for CES/PDREP and are not included in the RYG 
classification process. However, they do provide the 
contracting officer with a more complete profile of 
the contractor. [Ref. 11] 


The CES data base excludes: 


1. Material evaluations for base application and local 
use. 


2. Contractors developing major weapon systems. 


3. Medical procurement, material, suppliers, or 
evaluations. 


4. Subsistence procurement, material, suppliers, or 
evaluations. 


5. Unsatisfactory material condition caused by 
improper handling after receipt, deterioration during 
local storage, or inadequate maintenance or operation. 
6. Transportation discrepancies caused by the carrier. 


7. Ammunition and explosives accidents. 


8. Nuclear weapons procurement, material, suppliers, 
or evaluations. 


9. Naval Nuclear Power Plant primary system 
procurement, material, suppliers, or evaluations. 


10. Strategic Systems Project Office procurement, 
Suppliers, or material evaluations. [Ref. 3] 
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NMQAO utilizes the data base to classify the contractors 
according to the RYG program criteria and updates the RYG 
Status report monthly. 

The CES/PDREP identifies contractors whose quality history 
may require the use of additional pre-award or post-award 
quality assurance actions to ensure products of the required 
quality are received. However, under CES/PDREP there is no 
procedure to determine which offeror provides the best value 
to the Navy. Consequently, what makes the RYG concept unique 
is that it combines CES/PDREP contractor quality history with 
prescribed procedures to find the best value. RYG emphasizes 
contractor quality history by adding the cost of receiving 
poor quality goods or services into the procurement source 
selection process. RYG adds this cost to the offeror’s price, 
permitting the Contracting Officer to select a contractor on 


the basis of quality and cost, rather than cost alone. 


C. VALIDITY OF CES/PDREP DATA BASE 

A major concern of the RYG Program is the validity of the 
CES/PDREP data base and the effect that this possible lack of 
validity might have on contractor protests emerging from the 
RYG Program. To ensure that contractors have every 
opportunity to challenge specific classifications, NMQAO mails 


letters on a monthly basis to Red and Yellow classified 
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contractors detailing the reasons for their classification, 
the effect of the classification, and the procedures required 
to challenge the classification. During the test period, a 
total of 5,983 letters were mailed. Surprisingly, only 461 
responses were received, and of those only 109 were 
disagreements with the classification. Those challenges 
resulted in 53 corrections to the data base and 43 
classification changes. With less than 2% of all Red and 
Yellow classified contractors responding to the classification 
letter with challenges, and less than 1% of all contractors 
notified resulted in changes to the data base, the credibility 
of the data base has been firmly established. Furthermore, by 
sending notification letters to the contractors to inform them 
of their color classification and procedures for redress, 
NMQAO has virtually eliminated the possibility that protests 
based on the accuracy of the data base will be filed. Any 
contractor who fails to take timely action to correct these 
data base will be prevented under the rules of estoppel from 
utilizing the error in the data base as a basis for protest at 
a later date. CES/PDREP is updated monthly to include all 
corrective actions resulting from challenges and new 
information processed from all field activities. The 
Contracting Officer can then access the data base and from the 


classification and code assigned to the contractor, determine 
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whether a Technical Evaluation Adjustment (TEA) should be 


added to the contractor’s proposal. 


D. TECHNICAL EVALUATION ADJUSTMENT (TEA) 

Technical Evaluation Adjustment’s (TEA) are the 
anticipated additional costs the Government would incur for 
caries certain additional pre-award and post-award quality 
assurance actions when the contractor for that product is 
classified as "Red" or "Yellow". TEA’s are applied based on 
whether the award is considered a small purchase (< $25,000) 
or a major purchase (> $25,000). The procedures for applying 
the TEAs are as follows: 

1. For the purposes of the RYG program, simplified small 
purchase procedures were initially defined as purchases with 
a total value in excess of $2,500 but less than $25,000. When 
RYG procedures are used for simplified small purchases, the 
purchasing agent determines the offerors color classification 
from the data base and assigns the applicable standard TEA 
value as listed in APPENDIX C. The standard value is derived 
from the cost of additional quality assurance actions such as 
Government Source Inspection, Receipt Inspection, and Quality 
Assurance Letter of Instruction. The cost estimates of these 
quality assurance actions which are required to be performed 


are listed in APPENDIX D. The corresponding value assigned to 


ee, 


each of the quality assurance action are calculated as shown 
in APPENDIX E. 

2. For major purchases, RYG procedures require that the 
Contracting Officer determine the offeror’s colar 
classification and code from the RYG Evaluation Criteria 
listed in APPENDIX A. Utilizing the guidelines for TEA 
assignment in APPENDIX B, the Contracting Officer can 
determine which additional quality assurance requirements the 
Government will use. The additional requirements correspond 
to estimated costs listed in APPENDIX D. These costs have 
been computed from the standard costs listed in APPENDIX E. 
The total cost of the additional quality assurance 
requirements will give the Contracting Officer the required 
TEA. 

Except for actual DCMC costs, the estimated costs listed 
in APPENDIX D are provided as examples. Each activity must 
calculate its own set of TEA costs using the format in 
APPENDIX E, since the TEA costs are based on local prevailing 
test costs and labor rates. 

The TEA represents the anticipated cost to the Government 
to correct or take appropriate quality assurance action due to 
poor previous contractor performance. The application of the 
TEA raises a contractor’s proposed price. This provides the 


Contracting Officer with the ability in the source selection 
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process to obtain the supplies or services at the best overall 
value to the Government. 

After TEA’s have been computed and added to the 
contractor’s proposals, the contract is then awarded to the 
appropriate contractor. If the contract is subsequently 
awarded to a Green offeror, no other action is required. If, 
however, the contract is to be awarded to a Red or Yellow 
offeror, the Contracting Officer must insure that the 
appropriate clauses are added to the contract to ensure that 
additional quality assurance actions are taken during 


peLrlLormance. 


E. GREATEST VALUE/BEST BUY 

Another manner in which RYG is utilized is through 
GREATEST VALUE/BEST BUY evaluation criteria, which apply only 
to negotiated competitive solicitations. During the test 
period, the test activities developed evaluation plans and 
procedures tailored to their requirements. The evaluation 
plan considers price, which is given a minimum evaluation 
weight of 40%, and the remaining percentage apportioned only 
to quality. Point scores are then assigned according to the 
contractor’s RYG classification, and the offerors are then 
ranked according to those point scores for both factors (price 


and quality). 
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EF. SUMMARY 

This chapter described the background surrounding the 
Navy’s RYG Program. It introduced and briefly described the 
RYG Program evaluation criteria. It also described the 
CES/PDREP data base which is the basis of the RYG program. 
And finally, it explained the TEA and Greatest Value process 
of assigning adjustments to Red or Yellow contractors. The 
next chapter will analyze the performance of the RYG Program 


during the test period. 
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III. ANALYSIS OF RED/YELLOW/GREEN PROGRAM 


A. INTRODUCTION 

This chapter analyzes the results of the RYG Program Test 
to assess its ability to continue in an expanded environment 
(Navywide implementation). The chapter is organized in the 
following manner: 

1. The first section describes the capabilities of the 
computer systems which process data for the CES/PDREP and the 
RYG Program, and the data base which those systems manipulate. 
Furthermore, the size and accuracy of the data base and the 
ability of the computer systems to accommodate the increased 
amount of data to be generated from Navywide implementation is 
discussed. The means used by field activities to enter 
information into the CES/PDREP data base will also be 
addressed. 

2. The second section analyzes the performance of the RYG 
Program during the test period. This includes an analysis of 
the small and large purchase, and Greatest Value/ Best Buy 
procedures. 

BS. The third section analyzes the additional costs 


required to implement the RYG Program Navywide and the 


Za. 


anticipated savings to be achieved through the use of the RYG 
Program. 
4. The fourth section addresses the integration of the 
RYG Program with current acquisition systems and procedures. 
5% The last section discusses alternatives to the RYG 


Program. 


B. COMPUTER SYSTEMS AND DATA BASES 
1. Computer Capabilities 

The heart of the RYG Program is the computer system 
used to capture and process all the raw data for the 
CES/PDREP. The computer is a Univac 1100/73 mainframe located 
in Newport, Rhode Island and operated by Naval Computer and 
Telecommunications Station (NCTS), Newport, R.I. The computer 
operating system, data base management system, transaction 
interface package, query language, and other processing 
systems are all written by Univac for the 1100 series 
computer, and as such are proprietary in nature. The system 
contains standard main memory, tape and hard disk drive 
memory. [Ref. 12] 

There are 400 programs which drive the CES/PDREP 
information system. These programs are written in COBAL 
language. The current data base for CES/PDREP is 


approximately one gigabyte oor one billion bytes of 
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information. Since the data base is stored on magnetic tape 
and other external memory devices, the size of the data base 
ho Virtually Jimitiess. 

The time it takes to update the CES/PDREP data base is 
the major constraint imposed on the data base. Currently, the 
monthly update requires approximately 6 hours to complete. 
The computer system on which the CES/PDREP is processed 
adequately meets the additional data processing requirements 
that would be placed upon it with full Navywide 
implementation. [Ref. 11] 

The RYG Program data base is only 1.2 megabytes in 
size, as such it can be run on any IBM compatible computer. 
The minimum requirements for this microcomputer are at least 
640 kilobytes of Random Access Memory (RAM) and a hard disk 
drive memory with a minimum capacity of 10 megabytes. This 
means that even the oldest Personal Computers (PC) in the Navy 
inventory are able to run the RYG Program. 

2. Description of the Data Base 

The CES/PDREP data base is constantly updated with the 
addition of new information received from field activities. 
Once a month the classification program is run to update each 
supplier’s classification. The program generates reports and 
contractor notification letters. Contractor classification 


data from the CES/PDREP update are then used by NMQAO to 
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update the RYG Program. The RYG Program data base, which is 
used by participating activities, requires only 15 minutes to 
update after the monthly classification update by NCTS Newport 
is completed. 

The two major concerns associated with data base 
management are the size of the data base in relation to 
available memory and the accuracy of data input into the data 
base. In evaluating the feasibility of implementing the RYG 
Program, there are obvious concerns related to the anticipated 
size of the data base and the computer capabilities to handle 
that data. 

During the test period of the RYG Program, the five 
activities designated to participate in the program selected 
152 FSCs to which they would apply the RYG Program. 
Additionally, NMQAO collected data on all the FSCs which 
applied to Navy procurement. This amounted to approximately 
5,000 FSCs. The RYG data base currently contains information 
on more than 13,000 Contractors: It is anticipated that 
Navywide implementation will approximately quadruple the 
amount of input information resulting in a much larger data 
base. [Ref. 11] 

The size of the data base is a function of input data 


over which the RYG Program has limited control. However, 
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NMQAO is constantly investigating methods and procedures which 
will encourage activities to respond with pertinent data. 
Contractor information contained in the data base is 
the heart of the RYG Program. Therefore, the accuracy of the 
data base is essential for a successful program. A reasonable 
measure of the accuracy of the data base would be to analyze 


the responses received from contractors classified as Red or 


Yellow. As discussed in Chapter II,  NMQAO mails a 
notification letter to each contractor with these 
classifications after each monthly update. During the test 


period, 5,983 of these letters were mailed to contractors, 
with only 109 rebuttals to the classification assignment. This 
amounted to less than 2% of the total Red or Yellow classified 
contractors. Of these 109 responses, 53 resulted in changes 
to the data base, which resulted in only 43 classification 
changes. Whenever new data are received, the data base is 
immediately updated to reflect the change. In the event of a 
classification change, the contractor’s new classification was 
immediately communicated to all the test activities. 

Even though the above Sete eenOn that the data 
base is better than 99% accurate, NMQAO is continuing to 
implement stringent quality control measures to maintain the 
high accuracy rate. The majority of errors occur during the 


keypunch operation where the initial information is input into 
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the system. NMQAO has taken the current two step data input 
method used to update the CES/PDREP data base and attempted to 
streamline the process, while maintaining the data base 
integrity. In the current process, the raw data must flow 
from the field activities through their cognizant hardware 
systems command for initial review. For QDRs and RODs, a 
determination of contractor liability must be made at this 
level. The input data are further reviewed by Naval Air 
Systems Command in the case of QDRs or NMQAO for all other 
types of data, prior to its input into the CES/PDREP data 
base. 

The RYG Program is totally dependant upon the 
contractor quality information previously discussed in Chapter 
Il; however, without the necessary information with which to 
classify the contractors, the RYG Program cannot continue to 
function. To increase the amount and accuracy of contractor 
information flowing into the CES/ PDREP system, NMQAO has 
developed a program called Contractor Evaluation Data Entry 
System (CEDES). This program is a microcomputer based, menu 
driven, DBASE program designed to enable field activities to 
input the contractor quality information directly into the 
CES/PDREP data base. The program will allow the field 
activities to input pre and post award survey results, 


Material Inspection Records (MIRs), and Product Oriented and 
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Special Surveys directly into the CES/PDREP data base via dial 
up telecommunication lines. This program was initially 
designed to allow direct activity input while maintaining data 
accuracy. This is accomplished through the use of edit checks 
and validation tables available within the DBASE program. The 
program has just recently been deployed to five test sites for 
evaluation: 

1. The Naval Weapons Station, eee New Jersey 

2. Naval Shipyard Norfolk, Va. 

3. Naval Shipyard Charleston, S.C. 

4. Naval Air Engineering Center, Lakehurst, N.J. 

5. The Mare Island Naval Shipyard, Ca. 
The program has been in the field since 1 June 1991. 
Personnel training is being conducted and data input using 
CEDES is expected to start in July 1991. 

3. Summary 

In summary, the computers in use are capable of 
handling the additional information requirements which will 
come with Navywide implementation of the RYG Program. The 
data bases which are the basis of the RYG Program are 
accurate. NMQAO is investigating innovative methods to 
improve quality and reduce input barriers. The supporting 
system requirements are in place to facilitate Navywide 


implementation of the RYG Program. 
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C. ANALYSIS OF THE RYG PROGRAM TEST PERFORMANCE 

The CES/PDREP was developed in response to the Department 
of Defense and the Secretary of the Navy’s mandate that 
contractor quality history be maintained in a centralized data 
base. [Ref. 1] To make that organized data base useful, the 
RYG Program was developed and tested for one year ending on 30 
November 1990. This section addresses the performance of the 
RYG Program Test to assess the feasibility for Navywide 
implementation. 

1. RYG Program Procurement Procedures 

The RYG Program procurement procedures are divided 
into three types. 

a. Simplified Small Purchase. This was defined as all 
oral or written quotations resulting in awards with a total 
estimated value greater than $2,500 but less than $25,000. 
These purchases can include, at the discretion of the 
activity, Purchase Orders, Blanket Purchase Agreement (BPAs) 
calls, Imprest Fund purchases, and delivery orders against 
established Indefinite Delivery Indefinite Quantity contracts 
or General Services Administration (GSA) Federal Supply 
Services Contracts. [Ref. 13] Subsequently, the 
dollar threshold was lowered to zero to stimulate more 


activity for the test period. [Ref. 7] 
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b. Major Purchase. This is defined as all negotiated 
competitive solicitations for selected commodities with an 
expected contract value greater than $25,000. The use of 
major purchase procedures was limited to two of the five test 
Sites, SPCC, Mechanicsburg Pa., and NSC/NAD Pensacola Fl. 
(Ref. 14] 

c. Fixed Price/Greatest Value procedures. These are 
defined the same as those for major purchase, except that TEAs 
will not be used. Activities use a type of weighted 
evaluation criteria to evaluate the proposals as shown in 
APPENDIX F. The test activities established their own 
evaluation plan/procedures. These plans considered price and 
quality history as the only two evaluation factors. Price was 
given a minimum value of 40% with the remaining 60% applied to 
quality history. {Ref. 15] 

2. Test Period Results 

During the test period, there were 1,104 total RYG 
Program procurements of which 1,014 were competitive awards, 
62 were sole source awards, and 28 awards required that the 
RYG Program procedures be waived. Of the 1,014 awards, 631 
or nearly two-thirds of all the RYG awards went to 
"Insufficient Data" offerors. [Ref. 4] This large number of 
"Insufficient Data" awards can be attributed to the relative 


infancy of the CES/PDREP data base. 
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Of the 383 contract awards which were made to 
classified contractors, 55 resulted in displacements. A 
displacement occurs when the low offeror loses an award 
because of the TEA application. NMQAO has computed a 
displacement percentage by dividing the 55 displacements by 
the 383 RYG classified awards to give them a 14% displacement 
factor. This 14% figure excludes the insufficient data awards 
from the equation. As more quality history data becomes 
available, the majority of the insufficient data 
classifications will become a Green classification, thus 
raising the denominator and reducing the overall displacement 
rataor By using the total 1,014 RYG awards as the 
denominator, the displacement ratio becomes approximately 
55S": While this may be significantly lower than the 
publicized displacement rate of 14%, a displacement of 5% of 
the proven poor quality performers is still a considerable 
improvement over current practices. 

3. Reduced Number of Bids from Poor Performers 

An unexpected, yet welcome outcome of the RYG Program 
was the conspicuous absence from the competitive bidding 
process of several contractors with poor performance 
histories. This probably happened because they were made 
aware that the quality of their products were to be evaluated 


as a criteria for award. Interestingly, this phenomenon 
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occurred at all five test sites. While it is impossible to 
measure the cost savings of this phenomenon or even to 
determine why these poor quality contractors did not bid, 
their failure to bid is a potential intangible benefit to the 
Navy. If the threat of using past poor quality performance is 
enough to eliminate some of the worst offenders, the program 
is already paying for itself. 
4. Protests 

Another important factor is that there were only two 
protests lodged due to the use of RYG Program criteria. The 
first protest was made because the contract award went to 
other than the low offeror. The low offeror who was displaced 
lodged the protest on the basis that her company deserved 
special treatment because her firm is a "new, woman owned 
business". This protest did not challenge the red color 
Classification nor the criteria used to assign the color 
classification. The protest requested special dispensation 
because of minority ownership. This protest was received late 
and dismissed by the Contracting Officer. 

The second protest was made to the General Accounting 
Office (GAO) by a contractor who chose not to bid because of 
the RYG evaluation criteria. The contractor took issue with 


the fact that The RYG Program was used for this solicitation. 
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This contractor was currently on the Contractor Improvement 
Program. [Ref. 4] GAO dismissed the protest. 

In summary, the two protests filed did not challenge 
the RYG Program evaluation procedures or the assignment of 
color classification based upon the contractors past quality 
performance. 

5. Summary 

Although the RYG Program Test was very limited in 
nature and scope, it provided a valuable basis for evaluating 
the RYG Program and CES/PDREP reliability. The test’s 
restricted size enabled NMQAO, NAVSUP, ASN(RD&A) RM&QA, and the 
test activities to better manage this new program. 

This section has shown that the RYG Program is a 
significant improvement over current procedures. 
Additionally, the reduced bids from habitually poor performers 
demonstrated that the RYG Program will provide the Navy with 
improved product quality. Finally, the RYG Program evaluation 
criteria has a firm foundation in contractual law because the 
two protests that were filed were found in favor of the 


Government. 


D. ANALYSIS OF THE COSTS AND SAVINGS DUE TO IMPLEMENTATION 
This section analyzes the costs and savings of 


implementing the RYG Program Navywide. The costs include 
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software development, jetivity Sestabe-Up, Exainang ‘and 
personnel, and administrative actions. The savings include 
the reduced quality assurance actions. 
1. Cost of Implementation 
The costs associated with the RYG Program 
implementation are as follows: 
* Software development. 
Tench Miley Stare Up. 
x Training and Personnel. 
x Administrative actions. 
a. Software Development Costs 
The actual cost for the development of the software 
to allow RYG to be integrated with the APADE system could be 
defined through a statement of work if the Navy were to hire 
a contractor to develop the required programs. However, if 
the software was developed by Department of the Navy 
activities such as Fleet Material Support Office (FMSO), 
NMQAO, and NAVSUP, this software could be developed within a 
short period of time. [Ref. 11] The cost of development would 
be the salaries and overhead expenses associated with those 
personnel who worked on the project. These costs could be 
considered as sunk costs which would have been incurred with 


or without the project. Therefore, it is recommended that the 
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tasking be issued to NAVSUP to develop the software in 
cooperation with NMQAO and FMSO. 
b. Activity Start-up Costs 

Participation by a*"eontracting*activity in “the kee 
Program requires that they have a computer, with the 
capabilities listed previously in this chapter, for each 
purchasing agent or pair of agents involved in the Program. 
Initially, the only purchasing agents required to have the 
computers would be those whose specific task is related to 
material procurement, since RYG does not pertain to services. 
To further minimize initial costs, the RYG Program is 
compatible with the Navy standard Zenith 248 computers which 
are presently available at most contracting activities. The 
only additional piece of equipment required is 
telecommunications link (modem) with NMQAO in order to receive 
RYG Program updates. The cost of a top of the line 2400 baud 
modem with software is less than $150. 

c. Training and Personnel Costs 

Each contracting activity must be concerned with 
the impact of introducing a new program on its current 
workload and on its personnel. Although this cost is 
difficult to estimate in monetary terms, it is probably the 
Single most important concern expressed during interviews with 


contracting activity management personnel. 
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In these times of austere budgets and reductions in 
force, contracting activities are eager to find new programs 
or methods which will enhance efficiency and productivity 
without increasing the requirements placed on their work- 
force. It is perceived by contracting activities that there 
will be a high initial cost in terms of manpower investment 
associated with implementation of the RYG Program. However 
all initial investments in training and personnel costs will 
be overcome by the activity through the use of RYG Program. 

An example of reduced personnel costs would be if 
the contracting office received an offer from a contractor 
previously unknown to them who is, however, listed in the RYG 
Program as a Green contractor. The contracting activity would 
not require additional quality assurance actions, which most 
certainly would have been conducted when dealing with a new 
offeror. The only quality assurance required would be a query 
of the RYG Program. In this instance the contracting activity 
has saved the additional personnel costs of additional 
research on the offeror and additional quality assurance 
actions, such as a Quality Assurance Letter of Instruction. 

d. Administrative Costs 

What is perceived by most activities as a negative 

factor of the RYG Program is the potential for additional 


unbudgeted costs associated with quality assurance actions 
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required if a Red or Yellow offeror should win the award of a 
contract under the RYG Program. If a Red or Yellow offeror 
wins a contract award the unbudgeted costs are mostly 
administrative. These costs include the addition of the 
quality assurance clauses listed in APPENDIXES G, H, and I, 
and the issuance of Quality Assurance Letters of Instruction. 
These costs can be easily reduced, if not completely offset, 
by use of computer programs which will automatically include 
the required clauses and Quality Assurance Letters of 
Instruction in the contract. This program is already under 
development at NMQAO. 

The RYG Program also eliminates the need for 
unnecessary oversight of quality contractors and, instead, 
concentrates the oversight requirements on poor quality 
CONULraACEOES. 

2. Potential Implementation Savings 
Navywide implementation of the RYG Program could save 
the Navy Department approximately $39 million per year. Since 
implementation of the RYG Program Test, NMQAO has gathered the 
following data. 
Awards to Red and Yellow offerors with TEA: $7,799, Sie 
Awards to Red and Yellow offerors with avg. TEA: $7,913,743. 


The cost savings: $113,826. 
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The cost savings above resulted from the elimination 
of quality assurance actions for Green offerors and a reduced 
level of quality assurance actions for Yellow offerors. 
[Ref. 16] This cost savings represents 1.5% of the 
total RYG contracts awarded. 

Given the procurement actions for fiscal year 
1988/1989 as depicted in the Survey of Contracting Statistics, 
NAVSUP Publication 561, (APPENDIX J) an average of both years 
dollar value of the total Navy contracts is $8,623,705,000. 
However, this figure reflects all contract categories of which 
a certain portion are not applicable to the RYG Program, (i.e. 
service contracts, medical materials, and major weapons 
systems, etc.). 

NAVSUP (Code 026) estimates that only 30% of the total 
dollar value of all the Navy’s contracts are for material 
procurements which could apply to the RYG Program. This 
equates to approximately $2.6 billion. [Ref. 17] Le 
the Navy could save just 1.5% of the $2.6 billion, the total 
Savings attributable to the RYG Program would be approximately 
$39 million. This extremely conservative figure offsets costs 


incurred during further implementation of the RYG Program. 
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E.. INTEGRATION WITH APADE SYSTEM 

To most effectively and efficiently implement the RYG 
Program on a Navywide basis, it should be integrated with the 
existing APADE system. Most large Navy Field Contracting 
Activities use an automated system to run their procurement 
called the APADE system. The APADE system is run on a Tandem 
mini computer which uses an intelligent terminal. Each 
terminal is a PC in its own right, having a 640 kilobytes RAM, 
20 Megabyte hard disk drive, and a floppy drive. The 
operating system for these intelligent terminals is a Disk 
Operating System (DOS). Currently, the terminals have the 
capability to suspend the APADE program and access the 
programs stored in the hard disk drive. This allows the buyer 
to use programs for word processing or spreadsheets. This 
process does not involve exiting the APADE program but merely 
suspending its operation for the individual terminal. 

The RYG Program was designed to be compatible with the 
APADE hardware and operate from the intelligent terminal. If 
the RYG Program were installed onto the APADE terminals with 
the proper software interface to allow the user to toggle out 
of APADE and into the RYG Program, the buyer would have the 
flexibility to use the RYG Program with a minimum of effort 


and wasted time. 
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A major stumbling block to the integration of the APADE 
system and the RYG Program has been the difficulty of updating 
the RYG data base. A large contracting activity with 50 or 
more terminals would require an individual to update each 
terminal, resulting in many lost hours. However, with the 
proper interface, the update of the RYG data base can be 
accomplished in the same manner currently in place for use 
with the Enable program. The Enable program is a spreadsheet 
as part of the intelligent terminal and completely external of 
the APADE software. With the interface in place, an activity 
with the APADE system would receive the RYG update via modem. 
The update would then be loaded on the Tandem mini computer, 
and as each terminal is brought on line, the program interface 
would search for the update. Once located, the program would 
automatically process the update into the RYG data base stored 
in the terminal. 

The RYG Program was designed to be compatible with the 
existing APADE system. The software to allow these two 
systems to complement each other could be developed and 
installed within a minimum amount of time and at relatively 
low cost. This integration, while effective, expedient, and 
inexpensive, is still only a temporary solution. The ultimate 
goal must remain a totally integrated system of procurement 


where both systems will work together and function as one. 
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F. ALTERNATIVES TO RED/YELLOW/GREEN 

Currently, there are no other Navy alternatives which are 
ready for full implementation like the RYG Program. However, 
NAVSUP (Code 026), Director of Procurement Automation and 
Enhancement, is developing a contractor information system 
called the Buyer Information System (BIS). BIS is a new 
system which incorporates the basic RYG Program classification 
data base, General Services Administration (GSA) suspended and 
debarred list, Commercial and Government Entity (CAGE) files 
from Defense Logistics Studies Information Exchange (DLSIE), 
Navy Vendor Data Analysis Report (VDAR), and various business 
information from Dunn and Bradstreet. This new program takes 
the concept of tracking contractor quality history to a new 
level. While the RYG Program provides the Contracting Officer 
with a snapshot of the potential contractor’s past quality 
performance, the BIS provides a contractor profile by 
displaying 6 months of historical data. From this profile the 
Contracting Officer can then assess not only the current 
status of the contractor as with the RYG Program but also any 
trends in the contractor’s performance and financial 
Stability. The BIS Program is currently available to 
approximately 50 contracting activities through the 
Procurement Management Reporting System (PMRS). NAVSUP (Code 


026) estimates that to fully implement the BIS will require 
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approximately $1.6 million dollars for hardware procurement 
and software development. 

The BIS data base is 240 to 280 megabytes in size. 
Because of the amount of data contained in BIS, individual 
activities will require new computers with the 80386 chips and 
capable of having an expanded memory of 2 megabytes or 
weecacer. 

The BIS program has the potential to be an outstanding 
program, but full implementation of BIS is at least 5 years 
away. The RYG Program, on the other hand, is ready for 
implementation immediately. Additionally, implementation of 
the RYG Program would increase the contractor data base and 
thus provide better data to effectively start up the BIS 


program. 


G. SUMMARY 

This chapter analyzed the RYG Program elements, test 
performance, cost and savings, integration, and alternatives. 
The analysis of the test performance and cost savings shows 
the RYG Program to be a viable program with potential of 
saving the Navy millions of dollars a year. The research 
indicates that there are no Navy alternatives to the RYG 
Program which are available for immediate implementation. 


Finally, integration of the RYG Program with the APADE system 


4l 


has been identified as the most effective means of 
implementation. 
The next chapter addresses the impediments and benefits of 


implementing the RYG Program to determine whether 


implementation is feasible. 
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IV. IMPEDIMENTS AND BENEFITS OF IMPLEMENTATION 


A. INTRODUCTION 

This chapter discusses the impediments to and potential 
benefits of Navywide implementation of the RYG Program. The 
impediments and benefits discussed in this chapter resulted 
from information gathered from the following meetings and a 
SrLe Visite . 

1. The RYG Program Test wrap up meeting, held in 
Washington D.C., 5 March 1991. 

2. The Naval Air Systems Command briefing on the Quality 
Deficiency Report inputs into the CES/PDREP data base, held in 
Washington D.C.,. 6 March 1991. 

3. Site visit to NMQAO in Portsmouth N.H., 7 and 8 March 
oo 1. 

Additionally, information was gathered through personal 
and telephone interviews, RYG Program Test status reports and 


test site status reports. 


B. IMPEDIMENTS 
The impediments to Navywide implementation of the RYG 
Program must be considered in the initial stages of the 


program. Any single impediment, if severe enough, can 
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overshadow all potential benefits. This section will discuss 
the three most significant impediments to the successful 
implementation of the RYG Program: 

x Integration of the RYG Program with current 

procurement systems. 

x The perception of "Constructive Debarment" 

* Personnel resistance to the RYG Program. 

1. Lack of Integration of the RYG Program with Current 

Procurement Systems 
The lack of system integration is the strongest single 

argument from field activities against the RYG Program. The 
investment of scarce resources on a project which may not 
become an integrated part of the procurement system deters 
potential participants from implementing the RYG Program. 
Initially all activities which participated in the RYG Program 
Test and those interviewed as prospective participants in the 
program were very optimistic about the RYG Program ability to 
improve product quality (Ret. 7] However, based on the 
current inability of the RYG Program to integrate with APADE, 
which results in using two computer systems (one for the RYG 
Program and another for APADE), these same individuals rapidly 
lost their enthusiasm. For example, Portsmouth Naval Shipyard 
elected not to participate in the RYG Program until the system 


had been approved for integration with the APADE system. 
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[Ref. 18] The primary objection with this dual 
processing system were: 

a. The additional work load which would be placed upon 
their personnel resources. 

b: The additional start up costs that the activity may 
have to bear in order to make two computer systems 
available for the purchasing agents. 

In the current environment of constantly shrinking 
budgets and reduced manpower resources, a contracting activity 
must use its limited resources wisely. The implementation of 
the RYG Program without integration with APADE will create 
additional work for the contracting activity due to the 
increased operational and maintenance needs of a dual computer 
system. For example, the periodic maintenance of the RYG 
Program data base requires that monthly updates occur; 
however, without integration with APADE, this requires that 
each micro-computer data base be updated. In an integrated 
environment, these updates can be accomplished via software 
from the mainframe computer with a minimum of human 
intervention. 

The requirement to implement the RYG Program without 
the benefit of integration would require the contracting 
activity to invest funds to cover start-up costs. These costs 


reflect the need to buy additional hardware such as computers, 
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printers, and keyboards. This hardware will also require 
furniture such as desks and printer stands to support the new 
computer system. 

The practical solution to the funding issue is to 
integrate those systems. This will allow the contracting 
activities to avoid the cost of new hardware and support 
equipment. In addition, these activities would realize future 
Savings in the form of a more efficient and effective 
procurement workforce. Integration will result in purchasing 
agents having to use only one terminal, which has the ability 
to switch from APADE to the RYG Program, and to use one 
program for the automated application of TEAs. These factors 
help make an integrated program attractive to the field 
activities. 

2. The Perception of Constructive Debarment 

Constructive debarment is defined as the process by 
which an individual or group of contractors are prevented from 
bidding and/or winning an award of a contract without due 
process of law. The application of TEAs to an offeror’s 
proposal, which effectively raises the price of their bid, may 
be perceived by contractors as a form of constructive 
debarment. 

Under the RYG Program, a TEA is added to a Red or 


Yellow contractor’s offer when the Contracting Officer is 
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evaluating the offers. The TEA represents the expected cost 
to the Government of additional quality assurance measures. 
These measures must be taken to ensure a historically poor 
quality performer delivers material which conforms to the 
requirements in the contract. 

The RYG Program, in and of itself, does not 
constructively debar a company because after the application 
of the TEA, any contractor, including the ones with a Red 
classification, can still win the award if they continued to 
be the low responsible offeror. Additionally, if a Red or 
Yellow contractor does win the award, the Government will 
incur the costs represented by the TEA. 

The application of TEA does not prevent any contractor 
from competing and winning a contract. [Ref. 5] The TEA 
merely represents a cost to the Government of doing business 
with a Red or Yellow contractor. The cost is directly 
attributable to the contract, and to the past performance of 
the particular contractor. The intent of the RYG Program is 
not to debar Red or Yellow contractors, but to help insure 
that the Government receives the quality product it requires. 

During the test period, there were no protests citing 
constructive debarment as the reason. Additionally, during 
the planning phase of the RYG Program, the Office of General 


Counsel, Department of the Navy, presented the opinion that 
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the application of TEAs based upon a contractor’ syipast 
performance did not constitute constructive debarment. 
[Ref. 19] However, while investigating this area of concern, 
two situations were discovered where, through abuse of 
information contained in the RYG Program, a case for 
constructive debarment could be made. 

The first occurs when a Yellow contractor displaces a 
Red contractor and the Contracting Officer decides not to 
apply the appropriate quality assurance actions. This 
constructively debars the Red offeror from the competition 
because the TEA, which was used to displace the Red offeror, 
is now not part of the cost of the procurement. Since both 
offerors required some additional quality assurance action, 
the TEA must be enforced to ensure that the source selection 
process remains fair and impartial. 

The second instance occurs in the area of small 
purchase where a synopsis of the solicitation is not required. 
Contained in the current RYG Program is a sort routine by 
color classification which any activity can run to produce a 
list of all contractors contained in the RYG data base. A 
constructive debarment situation could occur when an activity 
uses a solicitation list made up of only contractors assigned 
the Green classification. This procedure constructively 


debars all Red and Yellow contractors from participation. 
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With the implementation of the RYG Program, the 
instructions which govern the program must be strengthened to 
include guidance that attempts to prevent these potential 
abuses. The guideline for the first situation should prevent 
the Contracting Officer from eliminating in contract 
administration the required quality assurance actions 
associated with the application of TEAs. While this guideline 
is already contained in the test procedure instructions, the 
Significance of this procedure was not clear and therefore 
must be emphasized. [Ref. 1] 

Further guidance must be provided on the use of the 
contractor classification list. This guidance would assist 
the procurement agents and help prevent the potential abuses 
of the RYG contractor information. The guidance must allow 
the contracting activity to produce a listing of all 
contractors of a particular commodity to be used on a bidder’s 
mailing list. However, it is recommended that the listing be 
provided in alphabetical order and without Color 
classifications. In this manner, the purchasing agent is able 
to rotate the solicitations without the possibility of 
constructively debarring a contractor. 

An additional solution to both the abuse of the RYG 
contractor information and the non-application of required 


quality assurance actions is to conduct a thorough training 


49 


session for each new site that implements the RYG Program. 
This training should be under the joint cognizance of NMQAO 
and NAVSUP. The training must include abuse awareness and 
consequences and legal modules to prevent the occurrences of 
constructive debarment actions. The training can be conducted 
in a classroom environment or through the use of video taped 
iInserucEaons - 
3. The Personnel Resistance to the RYG Program 

Another strong impediment to the RYG Program 
implementation is the resistance of contracting personnel to 
willingly embrace this new method of doing business. As with 
any new program, the RYG Program brings with it the initial 
disruption of normal daily operations. The implementation of 
the RYG Program requires contracting personnel to learn a new 
computer program, source selection evaluation methodology, 
procedures, and even a new vocabulary. Based on interviews 
and conversations with contracting activities, the 
implementation of the RYG Program is viewed by their personnel 
as being burdensome, rather than beneficial, because of 
disruption of normal operations, personnel training, 
comprehension, and instruction. These concerns stem from the 
lack of a clear understanding of the RYG concept and the 


procedures required by the program. 
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To successfully overcome this impediment, the managers 
of the implementation process must develop a plan that takes 
these concerns into account. An orientation and training 
program should be considered to ease the disruption of normal 
operation and provide a foundation for the acceptance and 
comprehension of the RYG Program. The personnel involved must 
be convinced that the implementation and use of the RYG 
Program will benefit the activities and the Government. 

4. Other Potential Impediments Considered 

Two additional concerns were brought up by contracting 
personnel who considered using the RYG Program: 

a. The requirement for a Certificate of Competency 
(CoC) from the Small Business Administration (SBA) when a 
small business is displaced and does not receive the award. 
The researcher discovered that prior to the initial RYG 
Program Test, the SBA and the Office of the General Counsel, 
Department of the Navy were consulted on this matter. The 
final decision was that a CoC was not required because 
application of a TEA as an evaluation factor was appropriate. 
The CoC dealt with the determination whether a contractor was 


responsible or not. It did not apply to the method of source 


selection. 
low The other area of concern was the possible 
resistance to the RYG Program by competing contractors. The 
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initial classification letters sent to the contractors were 
received with some resistance, particularly from the Red and 
Yellow classified contractors. These contractors, in turn 
were provided with the supporting documentation which drove 
the classification rating. Once this was done and a 
contractor was convinced that the system integrity was intact, 
the resistance dissipated. 

Since both of these potential impediments have been 
either solved by previous groundwork or failed to materialize 
as a major problem, these concerns are not potential 
impediments to the successful RYG Program implementation. 

5. Summary 

This section discussed the most important impediments 
that must be overcome if the RYG Program is to meet success 
when it implemented for use on a Navywide basis. The three 
impediments fell into the following broad categories: (1) 
computer hardware and software issues, (2) legal issues and 
(3) people issues. As a result of the research, a potential 
common solution to all the impediments is an implementation 
plan that incorporates mandatory education and training for 
all the personnel who will be involved with the program. The 
following section will discuss the benefits that could be 


gained by implementing the RYG Program. 
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Cc. BENEFITS 

This section will address the three most important 
benefits to be gained by fully implementing the RYG Program: 

* The reduction of quality assurance requirements. 

x Availability of the RYG Program for immediate use. 

* Availability of centralized automated information. 

The evaluation of the above benefits will revolve around 
the measurement of the benefits, the opposition to the 
benefits, and the ability of the benefit to overcome the 
impediments to the RYG Program. 

1. The Reduction of Quality Assurance Requirements 

The RYG Program helps assure the most efficient use of 
scarce resources. With the implementation of the RYG Program, 
the need for Government quality assurance requirements will be 
determined by the color classification of the contractor and 
the reasons for the classification (see APPENDIX A and B). 
The Contracting Officer will not have to make a decision 
concerning quality assurance requirements on every procurement 
subject to the RYG Program. 
The RYG Program uses predetermined requirements for 

quality assurance actions. This provides the contracting 
personnel with a road map to apply quality assurance actions 


when required and will significantly reduce time-consuming 
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efforts required when researching a contractor’s past 
performance. 

A contractor’s past performance history determines 
their RYG color classification. As such, the contractor is 
ultimately responsible for the quality assurance requirements 
the Navy imposes. The top-rated quality performers (Green 
contractors) will be free of the added burden of constantly 
having the Navy looking over their shoulder. 

This savings in costs associated with the reduction in 
quality assurance requirements has been measured since the 
beginning of the RYG Program Test. As seen in APPENDIX K, the 
cost avoidance savings through 28 February 1991, directly 
attributable to the reduction of quality assurance 
requirements is estimated to be in excess of $113,000. These 
Savings were the result of only five test site data and 152 
FSCs. 

The potential improvement in Government/contractor 
relationships is obviously more difficult to measure. 
However, given the current trend in the Department of Defense 
towards Total Quality Leadership (TQL), it is essential that 
this intangible benefit receive consideration. Responding to 
the tenets of TQL, the Navy should seek to foster trusting and 
businesslike relationships with its contractors. The 


elimination of unnecessary quality assurance actions for 
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proven quality contractors will help to bring this closer to 
reality. 

There is no opposition to the benefit derived from the 
reduction in quality assurance requirements. However, the 
measurement and the impact of the benefit is a matter for 
considerable debate. Even the most pessimistic view of this 
benefit acknowledges that the Navy will save money by the 
elimination of unnecessary quality assurance requirements. 

In the current environment of reduced spending, the 
fact that the RYG Program has demonstrated a potential to save 
millions of dollars is obviously the benefit which can help 
overcome the impediments previously discussed. 

2. Availability of the RYG Program for Immediate Use 

The RYG Program, although not perfect in every 
respect, is ready for Navywide implementation. The results of 
the test period have proven the program to be an effective 
method of source selection. This thesis has discussed several 
problems as well as suggestions to overcome them that will 
help make the implementation process more effective. Lo 
oppose the benefit of immediate use, there must be an 
available alternative to the RYG Program which provides a 
clear method of Navywide source selection improvement. None 
of the programs available use a Navywide data base to collect 


contractor quality data. These alternative programs use a 
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centralized data base similar to that of the RYG Program but 
lack the functional ability to translate the information into 
quality assurance actions. Furthermore, these alternative 
programs are years away from being available HOT 
implementation. 

The RYG Program is the only automated source selection 
improvement program which provides the Contracting Officer 
with a method of selecting a superior quality contractor. 
Furthermore, it is available for immediate implementation. 

3. Availability of Centralized Automated Information 

The RYG Program is capable of providing contracting 
activities Navywide with easy access to contractor quality 
performance data on Government contractors. This centralized 
automated data base not only provides contracting activities 
with up to date contractor performance data for use in source 
selection but it also allows them to use an FSC sorted listing 
to assist in finding sources. 

The RYG data base is very dynamic. As the RYG Program 
expands, more activities will submit information for the data 
base. This will improve the individual contractor quality 
histories as well as increase the number of contractors 
covered. The RYG Program is also dynamic in the sense that as 


new information is received the data base is updated to 
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reflect an accurate picture of the quality performance which 
can be expected from a contractor. 

With the advent of sophisticated computer networks and 
mainframe computers which have the ability to handle massive 
amounts of data, a truly integrated network data base has 
become possible. The RYG Program is the beginning of a 
network that eventually could come to be one of the most 
important information networks in the Navy. The RYG Program 
will put to practical use a data base which was under 
utilized. 

During interviews with test site contracting 
personnel, they expressed concern with the program’s monthly 
updates. They felt that more frequent updates were necessary 
to maintain the integrity of the program. The update process 
can be modified to meet the changing needs of the program. If 
more frequent updates become necessary, that procedure is 
easily implemented. 

The benefits derived from the availability of the 
centralized automated data base are not easily measured but 
the availability of the data base is an important link which 


makes the overall program more attractive for implementation. 


Se 


D. SUMMARY 

This chapter presented the impediments and benefits 
resulting from implementation of the RYG Program. The 
impediments discussed can not be ignored and all reasonable 
efforts should be undertaken to alleviate the concerns 
expressed. Dealing with these concerns will help facilitate 
the implementation process and create the best environment for 
success of the RYG Program. 

The final chapter will present the conclusions and 
recommendations of this thesis. The answers to the research 


questions will also be presented. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. ANSWERS TO THE RESEARCH QUESTIONS 
1. Should the Navy implement the RYG Program for all 
acquisition/procurement? 

The Navy should implement the RYG Program because 
Significant cost avoidance savings can result from its use. 
However, the RYG Program does not apply to all types of 
procurement. The RYG Program is best suited for use in 
procuring supply and not service type requirements. The 
specific federal supply classes that apply to the program have 
been determined and more are being considered. 

The source selection methodology being used by the RYG 
Program relies heavily upon the application of TEAS to 
determine the best qualified source. The results of this 
research indicated that use of TEAs is better suited for 
Simple and smaller dollar value procurements. Therefore, the 
RYG Program is best suited for use in small purchase 
procedures where the maximum award is less than $25,000. For 
those requirements over $25,000, it is the opinion of the 
researcher that the RYG Program is best suited for those 


procurements with an estimated value between $25,000 and 
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$100,000. At this value the application of the TEAs still 


have an effect on the source selection process. 


Din Is the versatility of the RYG Program sufficient to 


meet the requirements of Navywide procurement? 


This thesis has discussed the application of the RYG 
Program to small purchase, major purchase with TEA and Fixed 
Price/Greatest Value methods. Additionally, the RYG Program 
can be used as a stand alone program or integrated with the 
APADE system. While the RYG Program is not perfect for all 
procurements, it is versatile enough to meet the needs for the 
normal supply type procurements. 

3. Can the RYG Program be integrated with current 


procurement practices? 


The RYG Program was designed to meet current 
contracting practice requirements. The RYG Program has been 
reviewed by the Office of the General Counsel, Department of 
the Navy, prior to the initiation of the test period and found 
to be legally sound. 

The results of this research revealed that the RYG 
Program provides the Navy with an excellent method of 
improving source selection while continuing to provide the 
contractors with a fair opportunity to win the competition. 
RYG Program implementation will require that solicitations 


under the program stipulate that price and quality be used as 
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evaluation factors. This practice, while not used all the 
time, 1s still an accepted part of the procurement process. 
In conclusion, the implementation of the RYG Program will not 
require substantive changes in the current procurement 


practices and thus can easily be integrated. 


4. Can the current RYG Program be expanded to include 
all contractor quality history data for the entire 


Navy procurement system? 


The CES/PDREP and RYG Program data bases are compiled 
and manipulated by NMQAO. The computer upon which these 
programs are run is also used for many different applications. 
Because the current data are stored on external memory devices 
and archived into permanent storage files after one year, the 
RYG Program is limited only by the capability of the host 
computer. The current host computer system is sufficient to 
meet the computing needs of the RYG Program following full 


Navywide implementation. 


B. CONCLUSIONS 
The following conclusions were reached after the 
completion of the research effort for this thesis: 
1. The RYG Program should be implemented immediately. 
The RYG Program has successfully completed its initial 


test and is considered ready for Navywide implementation. 
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Other source selection improvement programs are still in their 
developmental phases and have not received permission to 
continue into a test period. While these programs may improve 
on the RYG Program concept with new and improved computers and 
software, they are still years and many millions of dollars 

away from full implementation. 
2. Implementation of the RYG Program could benefit the 
Navy by saving a significant amount of money in cost 


avoidance savings. 

Implementation of the RYG Program could result in 
significant savings to the Navy through cost avoidance. While 
the estimates of cost savings vary from as little as $38 
million to more than $440 million annually, both proponents 
and critics of the RYG Program do agree that there are 
Significant tangible savings to be obtained through 
implementation. The savings will result from the reduced 
quality assurance requirements, reduced cost for repair and 
return, and reduced reprocurement resulting from better 


quality material. 


3. The RYG Program should be integrated with the APADE 
System. 
The RYG Program as a stand alone program is very 


efficient as a source selection tool. However, for maximum 
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effectiveness and ease of implementation, the RYG Program 
should be integrated with the APADE system. By integrating 
the RYG Program with the APADE system, the contracting 
activity will have an automated procurement program and a 
source selection improvement program available in a functional 
and user friendly format on a single computer. 

4. The RYG Program is best suited for small purchases. 

The RYG Program source selection methodology relies on 
the application of the TEAs to displace poor quality 
performers. The TEA represents the cost to the Government of 
actual quality assurance requirements. The TEA for a 
particular color classification will remain the same whether 
the contract is for $5,000 or $500,000. The greatest impact 
and therefore the greatest benefit for the Navy will be on the 
lower valued requirements, where the possibility of 
displacement of the poor quality contractor is greater. 

5. The RYG Program should not be implemented for Navywide 
use in Fixed Price/Greatest value procurements. 

The Fixed Price/Greatest value portion of the RYG 
Program Test did not provide sufficient data to make an 
assessment of its value. With only two test sites using the 
Fixed Price/Greatest Value method on a limited number of 
procurements, more testing is required prior to a decision 


being made on its use. 
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C. RECOMMENDATIONS 


1. The Commander, Naval Supply Systems Command (NAVSUP) 
should take the lead and implement the RYG Program in 


the Navy Field Contracting System. 


The Naval Supply Systems Command is responsible for 
the Navy Field Contracting System that is made up of 
approximately 1000 commands with varying levels of contracting 
authority. NAVSUP is also responsible for the APADE automated 
procurement system; therefore, they are perfectly positioned 
to direct the implementation of the RYG Program. NAVSUP would 
provide a central point of contact for information, direction 


and training on the implementation of the RYG Program. 


2. NMQAO should assist NAVSUP with the development of the 
required training for the implementation of the RYG 
Program. 

NMQAO is responsible for the development '= and 
maintenance of the RYG Program and CES/PDREP data bases. In 
this capacity, NMQAO is the most qualified organization to 
provide the necessary training and technical assistance. In 
order for the implementation of the RYG Program to progress 
smoothly, a training program must be established to cover not 
only the RYG Program methods and procedures but also program 


benefits. The benefits, including the cost savings, reduced 
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workload, and reduction of oversight requirements, must be 
thoroughly understood to minimize the potential personnel 
resistance to the implementation of the RYG Program. 
3. The Navy should continue to research and develop new 
source selection improvement programs. 
The RYG Program provides the contracting officer with 
a very effective tool that can be used in the source selection 
process for supply type requirements. There are, however, 
many other types of requirements which the RYG Program does 
met, Cover. The implementation of RYG Program should not 
signify the end of other source selection improvement 
programs. Further research and development of source 


selection improvement programs should continue. 


D. AREAS FOR FURTHER RESEARCH 
There are three areas which require further research: 
i, BUYER INFORMATION SERVICE. 

Since the Buyer Information Service (BIS) is a new 
source selection improvement program under development, 
research to determine the merits of BIS is recommended. 

2. MATERIAL QUALITY. 

The material quality is the most difficult area of the 

RYG Program to measure. A baseline study of material reject 


rates for the five test sites which would include contracts 
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issued prior to and during the test period would be extremely 
valuable to determine the actual material quality improvement 
attributable to the RYG Program. 
3. DEPARTMENT OF DEFENSE IMPLEMENTATION OF A 
SOURCE SELECTION IMPROVEMENT PROGRAM. 
Another area of research would be to discuss and 
compare the different source selection improvement programs 


being used in the Department of Defense. 
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RED 


APPENDIX A: RED/YELLOW/GREEN EVALUATION CRITERIA 


CODE 


A 


B 


CODES AND DEFINITIONS 


EVALUATION CRITERION 
ON CURRENT NAVY VDAR 
METHOD C, D, AND/OR E CURRENTLY IN EFFECT 
QUALITY INFORMATION ON LATEST PRE-AWARD SURVEY 
(PAS) WITHIN LAST YEAR - NO AWARD 
LATEST PRODUCT-ORIENTED SURVEY (POS) IN LAST 
YEAR UNACCEPTABLE 
LATEST QUALITY SYSTEM REVIEW (QSR) IN LAST YEAR 
UNACCEPTABLE 
LATEST SPECIAL SURVEY IN LAST YEAR UNACCEPTABLE 
REJECT RATE 15% OR MORE IN LAST YEAR FOR 2 OR 
MORE LOTS 
LATEST TWO FIRST ARTICLE TESTS (FAT) IN LAST 
YEAR UNSATISFACTORY 
2 OR MORE CATEGORY "I" QDRS IN THE LAST YEAR 
6 OR MORE CATEGORY "II" ACTION QDRS IN THE LAST 
YEAR 
ON DLA CONTRACTOR ALERT LIST FOR MAJOR 


DEFICIENCIES 


oT 


YELLOW 


RED/YELLOW/GREEN EVALUATION CRITERIA 


CODES AND DEFINITIONS 


ISSUED VDAR LETTER OF CONCERN 

PREVIOUSLY CLASSIFIED "RED" - NOT WITHIN RED 
EVALUATION RANGE 

LATEST QUALITY PAS WITHIN LAST YEAR - AWARD 
WITH FINDINGS 

LATEST POS IN LAST YEAR ACCEPTABLE WITH 
CORRECTIONS 

LATEST QSR IN LAST YEAR ACCEPTABLE WITH 
CORRECTIONS 

LATEST SPECIAL SURVEY IN LAST YEAR ACCEPTABLE 
WITH CORRECTIONS 

REJECT RATE 6-14% FOR 2 OR MORE REJECTED LOTS 
IN LAST YEAR 

LATEST FAT IN LAST YEAR UNSATISFACTORY 

ONE CATEGORY "I" QDR IN LAST YEAR 

3-5 CATEGORY "II" ACTION QDRS IN LAST YEAR 

ON DLA CONTRACTOR ALERT LIST FOR MINOR 
DEFICIENCIES 

PREVIOUSLY RED - NO REJECTS FOR 5 OR MORE LOTS 


IN LAST 6 MONTHS 
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GREEN 


RED/YELLOW/GREEN EVALUATION CRITERIA 


CODES AND DEFINITIONS 


LATEST PAS IN LAST YEAR - AWARD WITH NO 
FINDINGS 

LATEST POS IN LAST YEAR ACCEPTABLE 

LATEST QSR IN LAST YEAR ACCEPTABLE 

LATEST SPECIAL SURVEY IN LAST YEAR ACCEPTABLE 
REJECT RATE LESS THAN 6% FOR 5 OR MORE LOTS IN 
LAST YEAR 

ALL FAT IN LAST YEAR SATISFACTORY 

O-2 CATEGORY "II" ACTION QDRS IN LAST YEAR AND 
G APPLIES 

PREVIOUSLY YELLOW - NO RBJECTS FOR 5 OR MORE 


LOTS IN LAST 6 MONTHS 
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APPENDIX B: 





CODE 
A la 
B la 
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D la 
E Ife 
FE la 
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N la 
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GUIDELINE FOR TEA ASSIGNMENT 


RED CLASSIFICATION 


ADDITIONAL QA REQUIREMENTS 


or Ib, 2a OF 2). 73 3:47 omCcno, 7. 


Or 


Or 


Or 


or 


Or 


Or 


Or 


or 


Or 


Or 


Or 


Or 


Or 


5 or 6 


5 0OL 6 


DivOr —6 


lb, 
lb, 
1b, 
1b, 
lb, 
lb, 
lb, 
1b, 
1b, 


i}8) 


2a 


Or 2 An 5S: (Omaae Gra 
5LOr 76,07, 

ne @ been ore 

Or 2D, 4g550r Gee 
5£Onr ome, 

45 5 © sGpe 

Or 2D, 4, 5.08 6,7 

ONG Aeye Sin lees) Oe on 
AVS OL 6,07 

orl2b, 3,44, S2omec ree 


YELLOW CLASSIFICATION 


1b, 
1b, 


ier 


3, 
3, 


4, 


4, 


4, 


5a Or 64 ..7 


Bo OL eG 7 


SaeOr ao 


70 


APPENDIX B 


GUIDELINE OF TEA ASSIGNMENT 





CODE ADDITIONAL QA REQUIREMENTS 
G AVS FOE 16) at 

H Za 4; 9 OF 6, 7 

J Ay 5: OP. 6; FT 

K A< to Or 6. 7 

N Lay, 4 eo OF 6 

P la cor jb; 4, 5 sor 6,7 7 


NOTE: The additional quality assurance actions depicted 
in this appendix are the RYG Program requirements. The 
abbreviations listed (ie: la or 2a) correspond to the quality 


assurance actions provided in APPENDIX D. 


fe 


APPENDIX C: SMALL PURCHASE TEAs 


RED CLASSIFICATION 
Government Source Inspection’ 
Receipt Inspection as Destination (Navy Rep)’ 


Quality Assurance Letter of Instruction”’ 


YELLOW CLASSIFICATION 
Government Source Inspection’ 


Quality Assurance Letter of Instruction’’ 


* Actual DCMC costs 
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$500* 
$1,194 


$755 
$2,449 


$500* 


$755 
S128 


APPENDIX D: MAJOR PURCHASE TEAs 


Quality Assurance Actions Estimated Cost 


1. Pre-Award Survey 
a. DCMC $500* 


b. PCO Representative Participation 


(Om loecal $775 
(2) Intermediate?’ $1,380 
(3) Distant? $2,095 


2. Post-Award Orientation 
a. DCMC $550* 


b. PCO Representative Participation 


(1) Local* Sa. 075 
(2) Intermediate” $2,110 
(Sy) bastant $3,590 


3. Product Oriented Survey (PCO Representative / DCMC) 


a. Local’ $800** 

b. Intermediate’® $1,500** 

Go Distant SZ2,215** 
4. Government Source Inspection’® $500* 


5. Receipt Inspection at Source (Navy and DCMC) 


a. Local $650**x 
b. Intermediate?’ $1,360*** 
Ga Distant: S262 * ex 


aS 


MAJOR PURCHASE TEAS 


6. Receipt Inspection at Destination (Navy) 


a. Low" $597 
b. Medium’? S14 
c. High’ $2 3a2 

7. Quality Assurance Letter of Instruction”’ S755x%x*x 


Notes: (1) Except for actual DCMC costs, as noted, the 
above costs are samples. Actual costs may vary between 
activities, based on each activity’s stabilized manhour rate. 

* actual DCMC cost 
x * includes actual DCMC cost - $400 


* x includes actual DCMC cost - $275 
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APPENDIX E: TEA CALCULATIONS 


1Calculated $30/hr x 8 hrs = $240 + $35 mileage = $275 + $500. 


2Calculated $30/hr x 8 hrs = $240 + $240 (8 hrs travel @ 
$30/hr)+ $200 (2 days per diem @ $100/day) + $200 travel costs 
= $880 + $500. 


3Calculated $30/hr x 8 hrs = $240 + $480 (16 hrs travel @ 
$30/hr) + $300 (3 days per diem @ $100/day) + $575 travel 
costs = $1,595 + $500. 


‘Calculated $30/hr x 16 hrs = $480 + $45 mileage = $525 + 
S55) 0 


SCalculated $30/hr x 16 hrs = $480 + $480 (16 hrs travel @ 
$30/hr) + $400 (4 days per diem @ $100/day) + $300 travel 
costs = $1,660 + $550. 


®Calculated $30/hr x 16 hrs = $480 + $960 (32 hrs travel @ 
$30/hr) + $600 (6 days per diem @ $100/day) + $1,000 travel 
costs = $3,040 + $550. 


’Calculated $30/hr x 12 hrs = $360 + $40 mileage = $400 + $400 
(DCAS costs). 


fealculatedad S30/hr x 12 hrs = $360 + $240 (8 hrs travel 6 
$30/hr) + $300 (3 days per diem @ $100/day) + $200 travel 
costs = $1,100 + $400 (DCAS costs). 


*Calculated $30/hr x 12 hrs = $360 + $480 (16 hrs travel @ 
$30/hr) + $400 (4 days per diem @ $100/day) + $575 travel 
costs = $1,815 + $400 (DCAS costs). 

Calculated $34.18/hr x 14 hrs. 


“Calculated $43/hr x 8 hrs = $344 + $31 mileage = $365 + $265 
(DCAS costs). 


’Calculated $43/hr x 8 hrs = $344 + $344 (8 hrs travel @ 


$43/hr) + $200 (2 days per diem @ $100/day) + $200 travel 
costs = $1,088 + $275 (DCAS costs). 


cS 


3Calculated $43/hr x 8 hrs = $344 + $688 (16 hrs travel @ 


$43/nr) + $300 (3 days per diem @ $100/day) + $575 travel 
costs = $1,907 + $275 (DCAS costs). 


“Calculated $43/hr x 4 hrs = $172 + $100 material handling + 
$325 test costs. 


Calculated $43/hr x 8 hrs 


$344 + $200 material handling + 
$650 test costs. 


Calculated $43/hr x 24 hrs = $1,032 + $500 material handling 
+ $800 test costs. 


“Calculated DCAS @ $34.18/hr x 8 hrs = $275 + $480 
(procurement representative @ $30/hr x 16 hrs). 
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APPENDIX F: FIXED PRICE-GREATEST VALUE SAMPLE EVALUATION 


Source Selection/Evaluation Method 
(The following example is illustrative of a source 
selection/evaluation method incorporated by RYG test 
procedures) 


Total Points (MAX) 
Total Technical 
Total Price 


100 points (%) 
60 points (%) 
40 points (%) 


CES Classification Technical Score 
Green 60 points 
Yellow 35 points 
Red 10 points 
Insufficient Data 60 points 


Price Score 


Within 0 percent- 5 percent of low offeror: GREEN 
Within 5+ percent-15 percent of low offeror: YELLOW 


Within 15 percent of low offeror: RED 
Green 40 points 
Yellow 26 points 
Red l37 pointes 


POSSIBLE OUTCOMES 


COLOR SCORE 
RANKING TECH PRICE TECH PRICE TOTAL SCORE 
i G G 60 40 100 
Z G Bs 60 26 86 
S x G 35 40 G2 
4 G R 60 ES 13 
) x zx 30 Z6 ol 
6 R G 10 40 50 
7 Bs R 35 TS 48 
8 R ea 10 ZS 36 
9 R R 10 FS 23 
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APPENDIX G: SIMPLIFIED SMALL PURCHASE CLAUSES 


NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 


(a) This procurement is subject to a test of the Navy’s 
Contractor Evaluation System (CES), "Red/Yellow/Green" (RYG) 
program. The test is authorized by the Assistant Sectary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 


(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractor’s past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. The 
price to be considered in determining best value will be the 
evaluated price after Technical Evaluation Adjustments (TEA) s 
for related quality assurance actions, as applicable, are 
applied to the offered price. 


(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTOR--TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 


(d) The commodities included in this test, as currently 
solicited, are: 


FSC No. FSC Nomenclature CLIN 





78 


ADDITIONAL EVALUATION FACTORS--TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (SIMPLIFIED SMALL PURCHASE PROCEDURES) 


(a) This procurement is part of a test of the Navy’s 
Contractor Evaluation System (CES) "Red/Yellow/Green" (RYG) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics), for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program’s effectiveness and impact on 
the acquisition process. 


(b) The purpose of RYG is to assist contracting personnel 
during source selection to determine the best value for the 
Government--price, past quality performance, and other factor 
considered. The test program uses accumulated contractor 
quality performance on selected commodities as either "Red" 
(high risk), "Yellow" (moderate risk), "Green" (low risk), or 
"Insufficient Data", based on the degree of risk to the 
Government of receiving poor quality products. Such 
Classifications are then used to apply Technical Evaluation 
Adjustments (TEA)s during source selection. 


(c) A TEA is a monetary assessment added to the price of 
selected commodities that have been classified as either 
"Red", or "Yellow" for specific contractors, and is based on 


the cost to the Government for effecting additional quality 
considerations that would otherwise not be required if award 
were made to a contractor with a satisfactory performance 
history. For purposes of requirements using the simplified 
small purchase procedures, standardized TEAs have been 
established for the "Red" and the"Yellow" classifications. 
During evaluation of quotations, the applicable TEA is added 
to the quoted price of the "Red" and/or "Yellow" commodity, 
and after consideration of any other pertinent price-related 
factors (e.g., transportation charges, First Article Testing, 
discount terms, etc.), becomes the basis for determining award 
of the purchase order. A commodity’s classification may 
change over time as new or revised quality performance data 
become available. 


(d) Classifications for the test program are summarized as 
follows: 
"Green"--Low risk. No extraordinary quality requirements or 


additional actions required; satisfactory quality history. 
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"Yellow"--Moderate risk. History of quality problems; special 
quality requirements/actions needed; Technical Evaluation 
Adjustments (TEA) applied to offered price. 


"Red"--High risk. Special alert to history of poor quality 
performance; TEA applied to offered price(s), and contract 
award requires higher level approval. 


"Insufficient Data"--Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, a TEA is not assessed. 


(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Naval Material Quality 
Assessment Office (NMQAO), Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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APPENDIX H: MAJOR PURCHASE CLAUSES 


NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 


(a) This procurement is subject to a test of the Navy’s 
Contractor Evaluation System (CES), "Red/Yellow/Green" (RYG) 
program. The test is authorized by the Assistant Sectary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 


(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractor’s past 
Quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. The 
price to be considered in determining best value will be the 
evaluated price after Technical Evaluation Adjustments (TEA)s 
for related quality assurance actions, as applicable, are 
applied to the offered price. 


cc) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTOR--TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 


(d) The commodities included in this test, as currently 
solicited, are: 


FSC No. FSC Nomenclature CLIN 
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ADDITIONAL EVALUATION FACTORS--TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (MAJOR PURCHASE PROCEDURES) 


(a) This procurement is part of a test of the Navy’s 
Contractor Evaluation System (CES) "Red/Yellow/Green" (RYG) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics), for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program’s effectiveness and impact on 
the acquisition process. 


(b) The purpose of RYG is to assist contracting personnel 
during source selection to determine the best value for the 
Government--price, past quality performance, and other factor 
considered. The test program uses accumulated contractor 
quality performance on selected commodities as either "Red" 
(high risk), "Yellow" (moderate risk), "Green" (low risk), or 
"Insufficient Data", based on the degree of risk to the 
Government of receiving poor quality products. Such 
classifications are then used to apply Technical Evaluation 
Adjustments (TEA)s during source selection. 


(c) A TEA is a monetary assessment added to the price of 
selected commodities that have been classified as either 
"Red", or "Yellow" for specific contractors, and is based on 
the cost to the Government for effecting additional quality 
considerations that would otherwise not be required if award 
were made to a contractor with a satisfactory performance 
history. During evaluation of quotations, the necessity for 
any additional quality assurance requirements will be 
determined, and the applicable TEA will be assessed onto the 
quoted price of the "Red" and/or "Yellow" commodity. After 
consideration of any other pertinent price-related factors 
(e.g., transportation charges, First Article Testing, discount 
terms, etc.), this adjusted price becomes the basis for 
determining award of the purchase order. A commodity’s 
classification may change over time as new or revised quality 
performance data become available. 


(d) Classifications for the test program are summarized as 
follows: 


"Green"--Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 
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"Yellow"--Moderate risk. History of quality problems; special 
quality requirements/actions needed; Technical Evaluation 
Adjustments (TEA) applied to offered price. 


"Red"--High risk. Special alert to history of poor quality 
performance; TEA applied to offered price(s), and contract 
award requires higher level approval. 


"Insufficient Data"--Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, a TEA is not assessed. 


(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Naval Material Quality 
Assessment Office (NMQAO), Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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APPENDIX I: FIXED PRICE - GREATEST VALUE CLAUSES 


NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 


(a) This procurement is subject to a test of the Navy’s 
Contractor Evaluation System (CES), "Red/Yellow/Green" (RYG) 
program. The test is authorized by the Assistant Sectary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 


(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractor’s past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. 


(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTOR--TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 


(d) The commodities included in this test, as currently 
solicited, are: 


FSC No. FSC Nomenclature CLIN 
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ADDITIONAL EVALUATION FACTORS--TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (FIXED PRICE--GREATEST VALUE PROCEDURES) 


(a) This procurement 1s part of a test of the Navy’s 
Contractor Evaluation System (CES) "Red/Yellow/Green" (RYG) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics), for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program’s effectiveness and impact on 
the acquisition process. 


(b) The purpose of RYG is to assist contracting personnel 
during source selection to determine the best value for the 
Government--price, past quality performance, and other factor 
considered. The test program uses accumulated contractor 
quality performance on selected commodities as either "Red" 
(high risk), "Yellow" (moderate risk), "Green" (low risk), or 
"Insufficient Data", based on the degree of risk to the 
Government of receiving poor quality products. A commodity’s 
classification may change over time as new or revised quality 
performance data become available. 


(c) For the purpose of source evaluation and selection, both 
the color classification of an offeror’s commodity and the 
proposed price(s) shall be evaluated in accordance with 
weighted evaluation criteria established by the Government 


prior to the receipt of proposals. Price-related factors, 
such as transportation charges, First Article Testing, 
discount terms, etc., will also be considered; however, no 


score or rating shall be applied. 


(d) Offerors are advised that, although price is of 
Significance in determining the successful offeror, past 
quality performance on the proposed commodity (as classified 
with the RYG data base) is essentially more important, and 
shall be evaluated accordingly. Each of the RYG 
classifications and its relative order of importance is 
Summarized as follows: 


"Green"--Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 
Commodities within this classification are apportioned a 
greater weight or value in the evaluation than those 
classified as either "Yellow" or "Red". 


85 


"Yellow"--Moderate risk. History of quality problems; special 
quality requirements/actions needed. Due to the additional 
quality assurance considerations that may be necessary, 
commodities within this classification are weighted less than 
those classified as "Green", but are of greater value than 
those within the "Red" category. 


"Red"--High risk. Special alert to history of poor quality 
performance; contract award requires higher level approval. 
These commodities are apportioned the least available weight 
or value for past quality performance relative to commodities 
within the "Green" or "Yellow" classifications. 


"Insufficient Data"--Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, commodities within this 
Classification shall be evaluated solely on the basis of price 
and related factors. Past quality performance will not be a 
consideration in the evaluation of commodities for which 
current quality performance data are not set forth within the 
data base. 


(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Naval Material Quality 
Assessment Office (NMQAO), Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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Procurement Actions Fiscal Years 1988/1989 


Contracting Statistics, NAVSUP Publication 561) 


ICPs 


NRCCs (less Naples) 


NSCs 


APPENDIX J: 


NAVAL LABS 


Miscellaneous 

NAS CORPUS CHRISTIE 
NAC INDIANAPOLIS 
MCAS CHERRY PT. 
LAKEHURST 

PAX RIVER 

POINT MUGU 


NAS 
NAS 
NAS 
NSY 
NSY 
NSY 
NSY 
NWC 
NOS 
NOS 
NOS 


Average for Activities during FY 88/89: 
woe 02 5, 705,000. 


NORFOLK 


PORTSMOUTH 
MARE ISLAND 
PEARL HARBOR 


CRANE 


INDIAN HEAD 
LOUISVILLE 


BAY ST. 
TOTAL 


Actions 


RYG COST AVOIDANCE CALCULATIONS 


iO eee 
2 7170 
420,568 
148,128 


5,196 
22,945 
19, 634 
10-167 
19,956 
ihe 752 
23,623 
Ls}. e ale 
16,519 

2,623 
19,700 

7,334 
10,896 


6,593 


S96, 293 


S (0008s) 
2, BOs, 2o0 
1-717 7039 
1,122,162 
1673-260 


4,866 
494,852 
25,213 
72276 
282,281 
35,536 
68, 032 
62,013 
40,899 
3,781 
138,408 
64,788 
63,879 


11,638 


G7 7490, 173 


RYG Test Displacement Rate - 14% 


Displacement during RYG test - 55 


Actions 
89,896 
25,159 

S55 .o7 7 

15357543 


5,114 
18,769 
15,288 
10,687 
18,119 
16,931 

9,810 
14, 980 
20427 
8,367 
18,640 
8,279 
10,258 


6,068 


806,512 


(From Survey of 


$ (000s) 
2,738,333 
1,462,077 
1,063,283 
1,944,414 


6,207 
326,717 
24,160 
115,976 
283,065 
49,410 
32,459 
47,837 
49,140 
15,564 
146,500 
133,114 
46,675 
13,305 


8,495 7256 


8527503 -ACeE1LOnS BLOT 


RYG DATA USING FY 88/89 FIGURES 


Competitive awards - low offeror is color classified -383 


OO mx 3(55/ 395) 


Estimated RYG Actions: 85,230 for $862,370,500. 


Average FY 88/89 actions for above sites: 852,303 for 
S§,6237705, 0007 


Estimated percentage subject to RYG - 10% 
85,230 =" 102 of "8527 see 


$862,370, 500 — 10¢ .ct 97523, 70, 008 


Estimated RYG Displacement Actions: 11,932 for $120,731, 870. 
RYG Test Displacement Rate - 14% 
Estimated _RYG actions. - 85,230 for $862,370,500. 
11,932 = 14% of 85,230 


9120,731,870 —"14t"ct 862,570, 5002 


Estimated Repair/Replacement Cost: $422,561,545. 


NOTE: Since RYG is not now operational at the sample 
implementation sites, the estimated 11,932 RYG 
displacement actions above represent awards to 
red or yellow low offerors. If half of these 
awards results in defective material, the cost 
of repairing/replacing the defective material 
is estimated to be seven times the cost of the 
material. 


Estimated operational RYG action dollars - $120,731, 870. 
S422,561,545..= 7 x (5 x $120,773 e704) 


Estimated Product Quality Deficiency Report (PQDR) Cost: 
$7,159,202 


NOTE: Since RYG is not now operational at the sample 
implementation sites, the estimated 11,932 RYG 
displacement actions above represent awards to 
red or yellow low offerors. If half of these 
awards results in defective material, Product 
Quality Deficiency Reports would be issued on 
each defective product. 
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Estimated RYG displacement actions - 11, 932 
PDQOR average preparation/processing cost - $1,200. 
Su ploy ZOO = polaZ00 ete tell, 932) 
Estimated additional Quality Assurance Actions Cost: 
$16,346, 840. 
Estimated RYG displacement Actions - 11,932 
Additional QA actions estimated cost - $1,370/action 


26,346,040 5= 51,370 x 11,932 
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APPENDIX K: CES RYG TEST STATUS REPORT 


Prepared 28 FEB 1991 


EXECUTIVE SUMMARY 
RED/YELLOW/GREEN PROCUREMENTS 


RYG Average 
Number Dollars (S$) PALT 
Total RYG Procurements ... 1668 
Waived -2.52.4 424 s.6 oe aLakal 
Awarded. 2.425 2-year. i psyei7/ 13757205, 306 50 
Lite Ae COMpPeEEIEdve: <2. aeee 1453 12,068,501 51 
b.>) Sole Source suse. 104 oD es es Oho: Cw 
It. a. S257000candwunder.. 515 1, 32373508 48 
Db. -OVEETS25,.0 Omer 42 5,882,425 aba lat 


Cost Comparisons 
(Competitive awards using TEAs. No Greatest Value/Best Buy 
Awards) 

1. Award to low offeror - with no TEAs: 


If all RYG procurement awards were to low 
offerors with no TEAS, Cost (co)... oe 71,5997226r 


2. Award to low offeror - with TEAs: 
If all RYG procurement awards were to low 


offeror with TEAS. €osts=(>))....... 71,913, 7432 
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Actual awards: 


The actual RYG procurement awards with TEAs 
for RED or Yellow awardee’s. Cost ($)..... 1797997217. 


Cost Avoidance: 
Cost avoidance is the low offerors price plus 


TEAS minus the awardee’s price plus TEAS (if any). 
Oi al eiaet Cay Pa ree ck ae er ene ae ee als 826. 


cal 


ASN 


BIS 


BPA 


CAGE 


CBD 


CEDES 


CES 


CLE 


COC 


DCMC 


DLA 


DLSIE 


DoD 


DON 


FAT 


FAR 


FMSO 


FSC 


GSA 


GSI 


APPENDIX L: GLOSSARY OF ACRONYMS 
Automation of Procurement and Accounting Data Entry 
Office of the Assistant Secretary of the Navy 
Buyer Information Service 

Blanket Purchase Agreement 

Commercial And Government Entity 

Commerce Business Daily 

Contractor Evaluation Data Entry System 
Contractor Evaluation System 

Contractor Improvement Program Alert List 
Certificate of Competency 

Defense Contractor Management Command 
Defense Logistics Agency 

Defense Logistics Studies Information Exchange 
Department of Defense 

Department of the Navy 

First Article Testing 

Federal Acquisition Regulation 

Fleet Material Support Office 

Federal Supply Classification 

General Services Administration 


Government Source Inspection 


me 


HM&E 
MIR 
MODEM 
NAVAIR 
NAVSEA 
NAVSUP 
NMQAO 
PAS 
reo 
PDREP 
PMRS 
POS 

QA 

QDR 
QSR 
ROD 
SBA 
SECNAV 
SECC 
SS 
Soro 
TEA 


VDAR 


Hull, Mechanical, and Electrical 
Material Inspection Record 
Modulator/Demodulator 

Naval Air Systems Command 

Naval Sea Systems Command 

Naval Supply Systems Command 

Navy Material Quality Assessment Office 
Pre-award Survey 

Procuring Contracting Officer 

Product Deficiency Report and Evaluation Program 
Procurement Management Reporting System 
Product-Oriented Survey 

Quality Assurance 

Quality Deficiency Report 

Quality System Review 

Report of Discrepancy 

Small Business Administration 

secretary of the Navy 

Navy Ships Parts Control Center 
Sub-Safe/Level 1 

Strategic Systems Project Office 


Technical Evaluation Adjustment 


Navy Vendor Data Analysis Report 
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